Estimating misclassification error with small samples via bootstrap cross-validation

Estimating misclassification error with small samples via bootstrap cross-validation

MOTIVATION Estimation of misclassification error has received increasing attention in clinical diagnosis and bioinformatics studies, especially in small sample studies with microarray data. Current error estimation methods are not satisfactory because they either have large variability (such as leave-one-out cross-validation) or large bias (such as resubstitution and leave-one-out bootstrap). W...

Cross-Validation and the Bootstrap: Estimating the Error Rate of a Prediction Rule

A training set of data has been used to construct a rule for predicting future responses. What is the error rate of this rule? The traditional answer to this question is given by cross-validation. The cross-validation estimate of prediction error is nearly unbiased, but can be highly variable. This article discusses bootstrap estimates of prediction error, which can be thought of as smoothed ve...

Estimating dimension From small samples

Dclails of a ncw algorirh~n lo cstimatc thc dirnc~~sion of a mcasurc tioru point s a m p l a arc clescribcd, T!ic ncw algorill~rn emphasizes rhal di~ncnsion ncctl no( be Il~e sarnc at dl scnlcs, 1l1a1 is, t l ~c r c ncctl not bc a "scaling rcgion. '~Ilc i\\gorilhm is ucriLcd with Gaussian o~c,~surcs and is found to bc rcliablc ancl hirvc Lcss domanding clnla rctl\~ircmcnts t h a n coavcn~ional ...

Error rate estimation via cross-validation and learning curve theory

Estimating Harrell’s Optimism on Predictive Indices Using Bootstrap Samples

In aging research, it is important to develop and validate accurate prognostic models whose predictive accuracy will not degrade when applied in external data sources. While the most common method of validation is split sample, alternative methods such as cross-validation and bootstrapping have some significant advantages. The macro that we present calculates Harrell’s optimism for logistic and...